install.packages("rpart")
Error in install.packages : Updating loaded packages
library(ROSE)
Loaded ROSE 0.0-3
Nessa atividade você irá usar seus conhecimentos sobre classificação para prever quais candidatos à Câmara de Deputados serão eleitos nas eleições de 2014. De forma específica faremos o seguinte:
1 Há desbalanceamento das classes (isto é, uma classe tem muito mais instâncias que outra)? Em que proporção? Quais efeitos colaterais o desbalanceamento de classes pode causar no classificador? Como você poderia tratar isso? (10 pt.) 2 Treine: um modelo de KNN, regressão logística, uma árvore de decisão e um modelo de adaboost. Tune esses modelos usando validação cruzada e controle overfitting se necessário, considerando as particularidades de cada modelo. (20 pts.) 3 Reporte precision, recall e f-measure no treino e validação. Há uma grande diferença de desempenho no treino/validação? Como você avalia os resultados? Justifique sua resposta. (10 pt.) 4 Interprete as saídas dos modelos. Quais atributos parecem ser mais importantes de acordo com cada modelo? (20 pts.) 5 Envie seus melhores modelos à competição do Kaggle. Faça pelo menos uma submissão. Sugestões para melhorar o modelo: (20 pts.) >> 1 Experimente outros modelos (e.g. SVM, RandomForests e GradientBoosting). >> 2 Experimente balancear as classes, caso estejam desbalanceadas. >> 3 Experimente outras estratégias de ensembles (e.g. Stacking)
Os dados estão neste link: https://www.kaggle.com/c/ufcg-cdp-20182-lab3/data (Links para um site externo)Links para um site externo
Para a entrega envie o link no RPubs e os arquivos .Rmd com o código em R. Para as respostas esperamos explicações textuais e visualizações para cada questão.
Setting up workspace
Loading DATA
Correlation
train.correlation %>%
select(-partido,
-uf,-grau,-sexo) %>%
na.omit() %>%
ggcorr(palette = "RdBu",
color = "grey50",
label = TRUE, hjust = 1,
label_size = 3, size = 4,
nbreaks = 5, layout.exp = 7) +
ggtitle("Gráfico de correlação eleições 2006")
data in column(s) 'ocupacao', 'situacao' are not numeric and were ignored

We choosed to remove those three categoric variables in order to run the model, otherwise it would take too much time. But for a better result you could let them on the data. And also remove those variable which have strong correlation
In the data would be better replace the NA for the column media, but we choosed replace by zero.
As we can see we have a considerable unbalance
Distribuition
#check classes distribution
prop.table(table(train$situacao))
eleito nao_eleito
0.1346103 0.8653897
FUTURE PLANS lets create a model and see how is goes whitout balance
accuracy.meas(test_data$situacao, pred.treeimb[,2])
Call:
accuracy.meas(response = test_data$situacao, predicted = pred.treeimb[,
2])
Examples are labelled as positive when predicted is greater than 0.5
precision: 0.955
recall: 0.949
F: 0.476
roc.curve(test_data$situacao, pred.treeimb[,2], plotit = F)
Area under the curve (AUC): 0.890
Lets balance it
Now the data set is balanced. But, you see that we’ve lost significant information from the sample. Let’s do both undersampling and oversampling on this imbalanced data. This can be achieved using method = “both“. In this case, the minority class is oversampled with replacement and majority class is undersampled without replacement.
table(data_balanced_both$situacao)
nao_eleito eleito
520 480
p refers to the probability of positive class in newly generated sample.
The data generated from oversampling have expected amount of repeated observations. Data generated from undersampling is deprived of important information from the original data. This leads to inaccuracies in the resulting performance. To encounter these issues, ROSE helps us to generate data synthetically as well. The data generated using ROSE is considered to provide better estimate of original data.
table(data.rose$situacao)
nao_eleito eleito
3842 3780
2 Treine: um modelo de KNN, regressão logística, uma árvore de decisão e um modelo de adaboost. Tune esses modelos usando validação cruzada e controle overfitting se necessário, considerando as particularidades de cada modelo. (20 pts.)
knn
model.knn
k-Nearest Neighbors
7622 samples
17 predictors
2 classes: 'nao_eleito', 'eleito'
Pre-processing: centered (30), scaled (30), remove (49)
Resampling: Cross-Validated (10 fold, repeated 10 times)
Summary of sample sizes: 6860, 6860, 6860, 6860, 6859, 6860, ...
Resampling results across tuning parameters:
k Accuracy Kappa
5 0.8123328 0.6237207
7 0.7994626 0.5978499
9 0.7884023 0.5756242
Accuracy was used to select the optimal model using the largest value.
The final value used for the model was k = 5.
usefull links: http://www.treselle.com/blog/handle-class-imbalance-data-with-r/ https://www.analyticsvidhya.com/blog/2016/03/practical-guide-deal-imbalanced-classification-problems/ https://shiring.github.io/machine_learning/2017/04/02/unbalanced
LS0tCnRpdGxlOiAiUiBOb3RlYm9vayIKb3V0cHV0OiBodG1sX25vdGVib29rCi0tLQoKYGBge3J9Cmluc3RhbGwucGFja2FnZXMoIm1sYmVuY2giKQppbnN0YWxsLnBhY2thZ2VzKCJDNTAiKQppbnN0YWxsLnBhY2thZ2VzKCJtYWdyaXR0ciIpCmluc3RhbGwucGFja2FnZXMoIlJPU0UiKQppbnN0YWxsLnBhY2thZ2VzKCJycGFydCIpCmBgYAoKYGBge3J9CmxpYnJhcnkoY2FyZXQpCmxpYnJhcnkobWxiZW5jaCkKbGlicmFyeShDNTApCmxpYnJhcnkoZHBseXIpCmxpYnJhcnkocGxvdGx5KQpsaWJyYXJ5KGNhcmV0KQpsaWJyYXJ5KFJPU0UpCmxpYnJhcnkocnBhcnQpCmxpYnJhcnkoR0dhbGx5KQpgYGAKCgpOZXNzYSBhdGl2aWRhZGUgdm9jw6ogaXLDoSB1c2FyIHNldXMgY29uaGVjaW1lbnRvcyBzb2JyZSBjbGFzc2lmaWNhw6fDo28gcGFyYSBwcmV2ZXIgcXVhaXMgY2FuZGlkYXRvcyDDoCBDw6JtYXJhIGRlIERlcHV0YWRvcyBzZXLDo28gZWxlaXRvcyBuYXMgZWxlacOnw7VlcyBkZSAyMDE0LiBEZSBmb3JtYSBlc3BlY8OtZmljYSBmYXJlbW9zIG8gc2VndWludGU6Cgo+PiAxIEjDoSBkZXNiYWxhbmNlYW1lbnRvIGRhcyBjbGFzc2VzIChpc3RvIMOpLCB1bWEgY2xhc3NlIHRlbSBtdWl0byBtYWlzIGluc3TDom5jaWFzIHF1ZSBvdXRyYSk/IEVtIHF1ZSBwcm9wb3LDp8Ojbz8gUXVhaXMgZWZlaXRvcyBjb2xhdGVyYWlzIG8gZGVzYmFsYW5jZWFtZW50byBkZSBjbGFzc2VzIHBvZGUgY2F1c2FyIG5vIGNsYXNzaWZpY2Fkb3I/IENvbW8gdm9jw6ogcG9kZXJpYSB0cmF0YXIgaXNzbz8gKDEwIHB0LikKPj4gMiBUcmVpbmU6IHVtIG1vZGVsbyBkZSBLTk4sIHJlZ3Jlc3PDo28gbG9nw61zdGljYSwgdW1hIMOhcnZvcmUgZGUgZGVjaXPDo28gZSB1bSBtb2RlbG8gZGUgYWRhYm9vc3QuIFR1bmUgZXNzZXMgbW9kZWxvcyB1c2FuZG8gdmFsaWRhw6fDo28gY3J1emFkYSBlIGNvbnRyb2xlIG92ZXJmaXR0aW5nIHNlIG5lY2Vzc8OhcmlvLCBjb25zaWRlcmFuZG8gYXMgcGFydGljdWxhcmlkYWRlcyBkZSBjYWRhIG1vZGVsby4gICgyMCBwdHMuKQo+PiAzIFJlcG9ydGUgcHJlY2lzaW9uLCByZWNhbGwgZSBmLW1lYXN1cmUgbm8gdHJlaW5vIGUgdmFsaWRhw6fDo28uIEjDoSB1bWEgZ3JhbmRlIGRpZmVyZW7Dp2EgZGUgZGVzZW1wZW5obyBubyB0cmVpbm8vdmFsaWRhw6fDo28/IENvbW8gdm9jw6ogYXZhbGlhIG9zIHJlc3VsdGFkb3M/IEp1c3RpZmlxdWUgc3VhIHJlc3Bvc3RhLiAoMTAgcHQuKQo+PiA0IEludGVycHJldGUgYXMgc2HDrWRhcyBkb3MgbW9kZWxvcy4gUXVhaXMgYXRyaWJ1dG9zIHBhcmVjZW0gc2VyIG1haXMgaW1wb3J0YW50ZXMgZGUgYWNvcmRvIGNvbSBjYWRhIG1vZGVsbz8gKDIwIHB0cy4pCj4+IDUgRW52aWUgc2V1cyBtZWxob3JlcyBtb2RlbG9zIMOgIGNvbXBldGnDp8OjbyBkbyBLYWdnbGUuIEZhw6dhIHBlbG8gbWVub3MgdW1hIHN1Ym1pc3PDo28uIFN1Z2VzdMO1ZXMgcGFyYSBtZWxob3JhciBvIG1vZGVsbzogKDIwIHB0cy4pCj4+Pj4gMSBFeHBlcmltZW50ZSBvdXRyb3MgbW9kZWxvcyAoZS5nLiBTVk0sIFJhbmRvbUZvcmVzdHMgZSBHcmFkaWVudEJvb3N0aW5nKS4KPj4+PiAyIEV4cGVyaW1lbnRlIGJhbGFuY2VhciBhcyBjbGFzc2VzLCAgY2FzbyBlc3RlamFtIGRlc2JhbGFuY2VhZGFzLgo+Pj4+IDMgRXhwZXJpbWVudGUgb3V0cmFzIGVzdHJhdMOpZ2lhcyBkZSBlbnNlbWJsZXMgKGUuZy4gU3RhY2tpbmcpCgpPcyBkYWRvcyBlc3TDo28gbmVzdGUgbGluazogaHR0cHM6Ly93d3cua2FnZ2xlLmNvbS9jL3VmY2ctY2RwLTIwMTgyLWxhYjMvZGF0YSAoTGlua3MgcGFyYSB1bSBzaXRlIGV4dGVybm8pTGlua3MgcGFyYSB1bSBzaXRlIGV4dGVybm8KClBhcmEgYSBlbnRyZWdhIGVudmllIG8gbGluayBubyBSUHVicyBlIG9zIGFycXVpdm9zIC5SbWQgY29tIG8gY8OzZGlnbyBlbSBSLiBQYXJhIGFzIHJlc3Bvc3RhcyBlc3BlcmFtb3MgZXhwbGljYcOnw7VlcyB0ZXh0dWFpcyBlIHZpc3VhbGl6YcOnw7VlcyBwYXJhIGNhZGEgcXVlc3TDo28uCgpTZXR0aW5nIHVwIHdvcmtzcGFjZQpgYGB7cn0Kc2V0d2QoIn4vZ2l0L2RhdGEtYW5hbHlzaXMvbGFiMDMvIikKYGBgCgoKTG9hZGluZyBEQVRBCgpgYGB7cn0KdHJhaW4gPC0gcmVhZC5jc3YoImRhdGEvYWxsL3RyYWluLmNzdiIpCnRlc3QgPC0gcmVhZC5jc3YoImRhdGEvYWxsL3Rlc3QuY3N2IikKYGBgCgpDb3JyZWxhdGlvbgoKYGBge3J9CnRyYWluLmNvcnJlbGF0aW9uMSA8LSB0cmFpbiAlPiUgc2VsZWN0KC1jKHNlcXVlbmNpYWxfY2FuZGlkYXRvLCBub21lLCBlc3RhZG9fY2l2aWwsIGFubywgY2FyZ28pKQp0cmFpbi5jb3JyZWxhdGlvbjEkc2l0dWFjYW8gPC0gYXMuZmFjdG9yKHRyYWluLmNvcnJlbGF0aW9uMSRzaXR1YWNhbykKdHJhaW4uY29ycmVsYXRpb24gPC0gdHJhaW4uY29ycmVsYXRpb24xICAlPiUKICBtdXRhdGUoc2l0dWFjYW8gPSBhcy5mYWN0b3Ioc2l0dWFjYW8pKSAlPiUKICBtdXRhdGUodWYgPSBhcy5mYWN0b3IodWYpKSAlPiUKICBtdXRhdGUocGFydGlkbyA9IGFzLmZhY3RvcihwYXJ0aWRvKSkgJT4lCiAgbXV0YXRlKHNleG8gPSBhcy5mYWN0b3Ioc2V4bykpICU+JQogIG11dGF0ZShncmF1ID0gYXMuZmFjdG9yKGdyYXUpKSAlPiUKICBtdXRhdGUob2N1cGFjYW8gPSBhcy5mYWN0b3Iob2N1cGFjYW8pKQoKdHJhaW4uY29ycmVsYXRpb24gJT4lIAogIHNlbGVjdCgtcGFydGlkbywKICAgICAgICAgLXVmLC1ncmF1LC1zZXhvKSAlPiUKICBuYS5vbWl0KCkgJT4lCiAgZ2djb3JyKHBhbGV0dGUgPSAiUmRCdSIsCiAgICAgICAgIGNvbG9yID0gImdyZXk1MCIsCiAgICAgICAgIGxhYmVsID0gVFJVRSwgaGp1c3QgPSAxLAogICAgICAgICBsYWJlbF9zaXplID0gMywgc2l6ZSA9IDQsCiAgICAgICAgIG5icmVha3MgPSA1LCBsYXlvdXQuZXhwID0gNykgKwogIGdndGl0bGUoIkdyw6FmaWNvIGRlIGNvcnJlbGHDp8OjbyBlbGVpw6fDtWVzIDIwMDYiKQpgYGAKCgpXZSBjaG9vc2VkIHRvIHJlbW92ZSB0aG9zZSB0aHJlZSBjYXRlZ29yaWMgdmFyaWFibGVzIGluIG9yZGVyIHRvIHJ1biB0aGUgbW9kZWwsIG90aGVyd2lzZSBpdCB3b3VsZCB0YWtlIHRvbyBtdWNoIHRpbWUuIEJ1dCBmb3IgYSBiZXR0ZXIgcmVzdWx0IHlvdSBjb3VsZCBsZXQgdGhlbSBvbiB0aGUgZGF0YS4gQW5kIGFsc28gcmVtb3ZlIHRob3NlIHZhcmlhYmxlIHdoaWNoIGhhdmUgc3Ryb25nIGNvcnJlbGF0aW9uCgpgYGB7cn0KdHJhaW4gPC0gdHJhaW4gJT4lCiAgc2VsZWN0KC1jYXJnbywgLW5vbWUsIC1vY3VwYWNhbywgLXRvdGFsX2Rlc3Blc2EsIC10b3RhbF9yZWNlaXRhLCAtc2VxdWVuY2lhbF9jYW5kaWRhdG8gKQp0ZXN0IDwtIHRlc3QgJT4lCiAgc2VsZWN0KC1jYXJnbywgLW5vbWUsIC1vY3VwYWNhbywgdG90YWxfZGVzcGVzYSwgLXRvdGFsX3JlY2VpdGEsIC1zZXF1ZW5jaWFsX2NhbmRpZGF0bykKYGBgCgpJbiB0aGUgZGF0YSB3b3VsZCBiZSBiZXR0ZXIgcmVwbGFjZSB0aGUgTkEgZm9yIHRoZSBjb2x1bW4gbWVkaWEsIGJ1dCB3ZSBjaG9vc2VkIHJlcGxhY2UgYnkgemVyby4KCgpgYGB7cn0KdHJhaW5baXMubmEodHJhaW4pXSA8LSAwCnRlc3RbaXMubmEodGVzdCldIDwtIDAKYGBgCgoKQXMgd2UgY2FuIHNlZSB3ZSBoYXZlIGEgY29uc2lkZXJhYmxlIHVuYmFsYW5jZSAKCgpgYGB7cn0KZGF0YV9jbGFzc19kZXN0cmlidXRpb24gPC0gdHJhaW4gJT4lIGdyb3VwX2J5KHNpdHVhY2FvKSAlPiUgc3VtbWFyaXplKGNsYXNzX2NvdW50ID0gbigpKQpwcmludChoZWFkKGRhdGFfY2xhc3NfZGVzdHJpYnV0aW9uKSkKYGBgCgpEaXN0cmlidWl0aW9uCgpgYGB7cn0KI2NoZWNrIGNsYXNzZXMgZGlzdHJpYnV0aW9uCnByb3AudGFibGUodGFibGUodHJhaW4kc2l0dWFjYW8pKQpgYGAKCgpgYGB7cn0KcCA8LSBwbG90X2x5KGRhdGFfY2xhc3NfZGVzdHJpYnV0aW9uLCB4ID0gfnNpdHVhY2FvLCB5ID0gfmNsYXNzX2NvdW50LCB0eXBlID0gJ2JhcicsCiAgICAgICAgbWFya2VyID0gbGlzdChjb2xvciA9IGMoJ3JnYmEoMjA0LDIwNCwyMDQsMSknLCAncmdiYSgyMjIsNDUsMzgsMC44KScpKSkgJT4lCiAgbGF5b3V0KHRpdGxlID0gIkNsYXNzIEJhbGFuY2UiLAogICAgICAgICB4YXhpcyA9IGxpc3QodGl0bGUgPSAiU2l0dWF0aW9uIiksCiAgICAgICAgIHlheGlzID0gbGlzdCh0aXRsZSA9ICJDb3VudCIpKQpwCgpgYGAKRlVUVVJFIFBMQU5TCmxldHMgY3JlYXRlIGEgbW9kZWwgYW5kIHNlZSBob3cgaXMgZ29lcyB3aGl0b3V0IGJhbGFuY2UKCmBgYHtyfQpzZXQuc2VlZCg0MikKaW5kZXggPC0gY3JlYXRlRGF0YVBhcnRpdGlvbih0cmFpbiRzaXR1YWNhbywgcCA9IDAuNywgbGlzdCA9IEZBTFNFKQp0cmFpbl9kYXRhIDwtIHRyYWluW2luZGV4LCBdCnRlc3RfZGF0YSAgPC0gdHJhaW5bLWluZGV4LCBdCgoKdHJlZWltYiA8LSBycGFydChzaXR1YWNhbyB+IC4sIGRhdGEgPSB0cmFpbl9kYXRhKQpwcmVkLnRyZWVpbWIgPC0gcHJlZGljdCh0cmVlaW1iLCBuZXdkYXRhID0gdGVzdF9kYXRhKQpuZXdfdGVzdCA8LSB0ZXN0CgpuZXdfdGVzdCA8LSB0cmFuc2Zvcm0ocHJlZC50cmVlaW1iLCBzaXR1YWNhbyA9IGlmZWxzZShlbGVpdG8gPiBuYW9fZWxlaXRvLCAiZWxlaXRvIiwgIm5hb19lbGVpdG8iKSkKCmFjY3VyYWN5Lm1lYXModGVzdF9kYXRhJHNpdHVhY2FvLCBwcmVkLnRyZWVpbWJbLDJdKQpgYGAKCgoKCgoKCmBgYHtyfQpyb2MuY3VydmUodGVzdF9kYXRhJHNpdHVhY2FvLCBwcmVkLnRyZWVpbWJbLDJdLCBwbG90aXQgPSBGKQpgYGAKCkxldHMgYmFsYW5jZSBpdAoKTm93IHRoZSBkYXRhIHNldCBpcyBiYWxhbmNlZC4gQnV0LCB5b3Ugc2VlIHRoYXQgd2XigJl2ZSBsb3N0IHNpZ25pZmljYW50IGluZm9ybWF0aW9uIGZyb20gdGhlIHNhbXBsZS4gTGV04oCZcyBkbyBib3RoIHVuZGVyc2FtcGxpbmcgYW5kIG92ZXJzYW1wbGluZyBvbiB0aGlzIGltYmFsYW5jZWQgZGF0YS4gVGhpcyBjYW4gYmUgYWNoaWV2ZWQgdXNpbmcgbWV0aG9kID0g4oCcYm90aOKAnC4gSW4gdGhpcyBjYXNlLCB0aGUgbWlub3JpdHkgY2xhc3MgaXMgb3ZlcnNhbXBsZWQgd2l0aCByZXBsYWNlbWVudCBhbmQgbWFqb3JpdHkgY2xhc3MgaXMgdW5kZXJzYW1wbGVkIHdpdGhvdXQgcmVwbGFjZW1lbnQuCmBgYHtyfQpkYXRhX2JhbGFuY2VkX2JvdGggPC0gb3Z1bi5zYW1wbGUoc2l0dWFjYW8gfiAuLCBkYXRhID0gdHJhaW4sIG1ldGhvZCA9ICJib3RoIiwgcD0wLjUsICAgICAgICAgICAgICAgICAgICAgICAgICAgICBOPTEwMDAsIHNlZWQgPSAxKSRkYXRhCnRhYmxlKGRhdGFfYmFsYW5jZWRfYm90aCRzaXR1YWNhbykKYGBgCgpwIHJlZmVycyB0byB0aGUgcHJvYmFiaWxpdHkgb2YgcG9zaXRpdmUgY2xhc3MgaW4gbmV3bHkgZ2VuZXJhdGVkIHNhbXBsZS4KClRoZSBkYXRhIGdlbmVyYXRlZCBmcm9tIG92ZXJzYW1wbGluZyBoYXZlIGV4cGVjdGVkIGFtb3VudCBvZiByZXBlYXRlZCBvYnNlcnZhdGlvbnMuIERhdGEgZ2VuZXJhdGVkIGZyb20gdW5kZXJzYW1wbGluZyBpcyBkZXByaXZlZCBvZiBpbXBvcnRhbnQgaW5mb3JtYXRpb24gZnJvbSB0aGUgb3JpZ2luYWwgZGF0YS4gVGhpcyBsZWFkcyB0byBpbmFjY3VyYWNpZXMgaW4gdGhlIHJlc3VsdGluZyBwZXJmb3JtYW5jZS4gVG8gZW5jb3VudGVyIHRoZXNlIGlzc3VlcywgUk9TRSBoZWxwcyB1cyB0byBnZW5lcmF0ZSBkYXRhIHN5bnRoZXRpY2FsbHkgYXMgd2VsbC4gVGhlIGRhdGEgZ2VuZXJhdGVkIHVzaW5nIFJPU0UgaXMgY29uc2lkZXJlZCB0byBwcm92aWRlIGJldHRlciBlc3RpbWF0ZSBvZiBvcmlnaW5hbCBkYXRhLgoKYGBge3J9CmRhdGEucm9zZSA8LSBST1NFKHNpdHVhY2FvIH4gLiwgZGF0YSA9IHRyYWluLCBzZWVkID0gMSkkZGF0YQp0YWJsZShkYXRhLnJvc2Ukc2l0dWFjYW8pCmBgYAoKCj4+IDIgVHJlaW5lOiB1bSBtb2RlbG8gZGUgS05OLCByZWdyZXNzw6NvIGxvZ8Otc3RpY2EsIHVtYSDDoXJ2b3JlIGRlIGRlY2lzw6NvIGUgdW0gbW9kZWxvIGRlIGFkYWJvb3N0LiBUdW5lIGVzc2VzIG1vZGVsb3MgdXNhbmRvIHZhbGlkYcOnw6NvIGNydXphZGEgZSBjb250cm9sZSBvdmVyZml0dGluZyBzZSBuZWNlc3PDoXJpbywgY29uc2lkZXJhbmRvIGFzIHBhcnRpY3VsYXJpZGFkZXMgZGUgY2FkYSBtb2RlbG8uICAoMjAgcHRzLikKCiNrbm4KCmBgYHtyfQpmaXRDb250cm9sIDwtIHRyYWluQ29udHJvbChtZXRob2QgPSAicmVwZWF0ZWRjdiIsIAogICAgICAgICAgICAgICAgICAgICAgICAgICBudW1iZXIgPSAxMCwKICAgICAgICAgICAgICAgICAgICAgICAgICAgcmVwZWF0cyA9IDEwKQoKcHJlUHJvY2VzcyA9IGMoImNlbnRlciIsICJzY2FsZSIsIm56diIgKQo/IHRyYWluCm1vZGVsLmtubiA8LSB0cmFpbihzaXR1YWNhbyB+IC4sIAogICAgICAgICAgICAgICBkYXRhID0gZGF0YS5yb3NlLAogICAgICAgICAgICAgICB0ckNvbnRyb2wgPSBmaXRDb250cm9sLAogICAgICAgICAgICAgICBtZXRob2QgPSAia25uIiwgIyBwb2RlIHNlciAnbGFzc28nbGRmCiAgICAgICAgICAgICAgIG1ldHJpYyA9ICJBY2N1cmFjeSIsCiAgICAgICAgICAgICAgIHByZVByb2Nlc3MgPSBwcmVQcm9jZXNzKQoKbW9kZWwua25uCmBgYAoKCnVzZWZ1bGwgbGlua3M6Cmh0dHA6Ly93d3cudHJlc2VsbGUuY29tL2Jsb2cvaGFuZGxlLWNsYXNzLWltYmFsYW5jZS1kYXRhLXdpdGgtci8KaHR0cHM6Ly93d3cuYW5hbHl0aWNzdmlkaHlhLmNvbS9ibG9nLzIwMTYvMDMvcHJhY3RpY2FsLWd1aWRlLWRlYWwtaW1iYWxhbmNlZC1jbGFzc2lmaWNhdGlvbi1wcm9ibGVtcy8KaHR0cHM6Ly9zaGlyaW5nLmdpdGh1Yi5pby9tYWNoaW5lX2xlYXJuaW5nLzIwMTcvMDQvMDIvdW5iYWxhbmNlZCAKCgoK